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Multi-carri r Watermarks Using Carrier 
Signals Modulated with Auxiliary Messages 

Related Application Data 

This application is a continuation of US Patent Application No. 09/619,266, filed July 
5 19, 2000, which is hereby incorporated by reference. This application is also related to US 
Patent Application No. 09/503,881, filed February 14, 2000, which is hereby incorporated by 
reference. 

Technical Field 

10 The invention relates to encoding and decoding messages in media such that changes 

resulting from the encoding process are imperceptible to humans yet recoverable by an 
automated decoding process. 

Background and Summary 

Digital watermarking is a process for modifying media content to embed a machine- 
15 readable code into the data content. The data may be modified such that the embedded code 
is imperceptible or nearly imperceptible to the user, yet may be detected through an 
automated detection process. Most commonly, digital watermarking is applied to media such 
as images, audio signals, and video signals. However, it may also be applied to other types 
of data, including documents (e.g., through line, word or character shifting), software, multi- 
20 dimensional graphics models, and surface textures of objects. 

Digital watermarking systems have two primary components: an encoder that embeds 
the watermark in the media content, and a decoder that detects and reads the embedded 
watermark. The encoder embeds a watermark by altering data samples of the media content. 
The reading component analyzes content to detect whether a watermark is present. In 
25 applications where the watermark encodes information, the reader extracts this information 
from the detected watermark. 
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The invention provides a system and related methods for encoding and decoding 
watermarks from media signals, including image and audio signals, using multiple carriers. 
A multi-carrier watermark system includes a watermark embedder and compatible decoder. 
The embedder encodes auxiliary information, such as a binary message, into the host media 
5 signal by modulating message signals with two or more corresponding carrier signals to form 
a watermark signal and embeds this signal into the host signal. A compatible watermark 
decoder uses the carrier signals to demodulate the message signals from the watermarked 
signal. 

Further features of the invention will become apparent with reference to the following 
1 0 detailed description and accompanying drawings. 



Brief Description of the Drawings 

Fig. 1 is a flow diagram illustrating a multi-carrier watermark embedder. 

Fig. 2 is a flow diagram illustrating a multi-carrier watermark decoder. 
1 5 Fig. 3 illustrates an FFT of a single carrier signal and 4 carrier signals. 

Fig. 4 illustrates an example of modulation functions corresponding to the 4 carrier 
signals shown in Fig. 3. 

Fig. 5 shows an example of possible watermark signal waveforms generated using the 
modulation functions of Fig. 4 and message signals in binary antipodal form. 
20 Fig. 6 illustrates a reduced set of possible waveforms when the watermark signal is 

confined to a binary antipodal form. 

Fig. 7 shows an example of an odd-parity, multi-carrier watermark embedder. 

Fig. 8 shows a watermark message demodulator compatible with the embedder of 

Fig. 7. 

25 Fig. 9 is a diagram of an alternative multi-carrier watermark embedder. 

Fig. 10 is a diagram of a watermark demodulator compatible with the embedder 
shown in Fig. 9. 

Fig. 1 1 is a diagram of a watermark embedder in which modulation functions are 
depicted generally. 
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Fig. 12 depicts a specific example of modulation functions that may be employed in 
the embedder shown in Fig. 1 1 . 

Fig. 13 is a diagram showing an evaluation of the modulation functions of Fig. 12. 

Detailed Description 

5 Introduction 

The following sections describe watermark encoders and decoders that employ 
multiple carriers. The encoder modulates message signals with a set of carriers and encodes 
the modulated messages into a media signal, such as audio, image or video signal. Using the 
known carrier signals, the decoder demodulates the modulated messages from a signal 
1 0 suspected of containing a watermark. 

Fig. 1 is a flow diagram illustrating a multi-carrier watermark embedder. A 
modulator 100 modulates message signals 102 with corresponding carrier signals 104. In 
this multi-carrier watermarking scheme, the watermark signal, w, is a sum of message signals 
(fO, fl, f2, f3) modulated by corresponding carriers (cO, cl, c2. . .). 
15 w = f0c0 + flcl + f2c2 + f3c3... 

An encoder 106 then encodes the modulated message signals into a host signal 108 to 
create a watermark signal 110. 

Fig. 2 is a flow diagram illustrating a multi-carrier watermark decoder. The input is a 
signal 200 suspected of containing a watermark. The decoder predicts the original signal 
20 202, and removes it to compute an estimate of the watermark signal. While this prediction 
operation is useful to increase the watermark signal to noise ratio, it is not necessary. A 
demodulator 204 then demodulates message signals 206 from the watermark signal using the 
carrier signals 208. 

To illustrate, consider an example of a watermarking scheme for embedding an 
25 auxiliary signal into a two-dimensional host signal, such as an image. Fig. 3 illustrates an 

FFT of a single carrier signal and 4 carrier signals. In this example, the resolution of the host 
image is 100 dpi, which provides a bandwidth up to 50 cycles per inch. Fig. 3 shows how the 
available bandwidth up to 50 cycles per inch is divided for examples of a single carrier, and 
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four carriers. The single carrier is at DC, while the four carriers are located at DC, and 50 
cycles per inch in horizontal, vertical and diagonal dimensions. The 4 carriers are chosen to 
equally divide the available bandwidth. This is not a requirement, however. 

In the example of four carriers, each carrier can be represented as a modulation 
5 function as shown in Fig. 4. In the digital domain, where continuous functions are sampled 
(e.g., pixels in a digital image are examples of discrete samples), the modulation functions 
can be represented by sample sequences as shown in Fig. 4. Because the bandwidth allotted 
to each carrier is equal to one-half the total bandwidth in the x and y directions, we need only 
one-half the spatial resolution to represent each band. Stated alternatively, each message 

10 sample for the carriers is modulated to 2 by 2 image samples. Larger or smaller bandwidth 
divisions will require fewer or more image samples on average, respectively. 

Referring again to Fig. 4, a watermark signal at each 2 by 2 block is formed by 
multiplying message sample values (e.g., 1, -1) with corresponding carrier signals and 
summing the results for each sample. 

15 In the case where the message sample values are 1 or -1, the possible waveforms for 

a 2 by 2 block are shown in Fig. 5. Note that these waveforms include five possible states {- 
2, -1 , 0, 1 , and 2} . The waveforms may be applied to the host signal using a variety of 
encoding functions, which are well documented in literature on digital watermarking. 

One form of signaling that is particularly useful in watermark encoding is Binary 

20 Phase Shift Keying (BPSK). In this form of signaling, the watermark signal may be 

represented as a binary anti-podal signal consisting of two states: one and negative one. 

The possible waveforms shown in Fig. 5 can be reduced to a set of binary antipodal 
signals by removing the waveforms that have more than the states of one and negative one. 
For our running example, this reduces the possible set of waveforms from 16 to 8 as shown 

25 in Fig. 6. Observing the remaining waveforms, one can see that there needs to be odd parity 
between the input message samples, fO, fl, £2, f3. One can arbitrarily select one of the inputs 
to be the lost degree of freedom due to the odd parity requirement. 

Fig. 7 shows an example of an odd-parity embedder. This embedder employs 
multiple carrier signals, yet generates a binary antipodal watermark signal. Each of three 

30 input message symbols are applied to a spreading function to generate an intermediate input 
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(eO, el, e2). From these intermediate inputs, the embedder computes the value of a parity bit. 
The intermediate input values, along with the parity bit, form a set of message sample inputs. 
These message sample inputs are modulated by corresponding carrier signals and then 
summed to form a binary antipodal watermark signal. 
5 Fig. 8 shows a watermark message demodulator compatible with the embedder of 

Fig. 7. A binary antipodal watermark signal W is input to the demodulator. This signal is 
demodulated with the carrier functions. The intermediate results from three of the 
demodulation operations are applied to an inverse of the spreading function to compute the 
message symbols fO, fl, and f2. 

10 Fig. 9 is a diagram of an alternative multi-carrier embedder. This embedder allows 

for two inputs fO, fl, rather than three as in the previous example. This embedder encodes 
two bits per symbol (fO and fl) and enforces odd parity on one and even parity on the other. 
In Fig. 9, the bit values for each input are represented as bO, bl, b2 and b3. Parity control 
bits p0, pi, p2 and p3 are set such that one and only one is set to -1. The bit values are 

1 5 multiplied by the parity control bit and then applied to the spreading function, which is an 
exclusive OR (XOR) operation.. The outputs mO, ml, m2 and m3 are modulated by 
corresponding carrier signals C0,0, C1,0, Cl,l and C0,1. The resulting modulated signals are 
summed to form a watermark signal W. 

Carrier assignment may be random by position (e.g., x, y coordinates) in the host 

20 signal. Such random assignment applies if f0,n or fl,n are repeated a number of times in the 
image. For example, in some implementations, the message is spread over a pseudorandom 
number and repeated throughout the host signal. When the message samples are repeated, 
the instances of the same repeated message samples do not always have to be assigned to the 
same carrier signals. 

25 Fig. 10 is a diagram of a watermark demodulator compatible with the embedder 

shown in Fig. 9. As depicted in the diagram, the decoding operations reverse the embedding 
operations. Starting with the watermark signal W (or an estimate of it), the demodulator 
demodulates the watermark signal with the carrier functions. It then applies the inverse of 
the spreading function. The resulting values are then multiplied by the parity control bits, 

30 and summed to re-construct the message signals. 
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The example demodulators shown in Figs. 8 and 10 both apply a demodulation 
operation on the watermark signal W with a set of carrier signals. A number of different 
filters may be used to implement these demodulation operations. One approach is to use a 
set of matched filters corresponding to each of the carrier signals. For example, a matched 
5 filtering operation computes the dot product of the sampled carrier signal with the 

watermarked or estimated watermark signal (e.g., prediction is used to compute the estimated 
watermark signal from the watermarked signal). In this approach, the demodulator performs 
a matched filtering between the watermarked or estimated watermark signal and each of the 
carrier signals. 

10 An alternative approach is to predict the original, un- watermarked signal sample 

values, derive an estimate of the watermark signal values using the predicted values (e.g., 
subtracting the predicted from the watermarked signal), and then interpret the meaning of the 
estimated watermark signal using the set of possible waveforms, for example, in a look up 
table of sample polarities (e.g., matching the estimated watermark waveform with the set of 

15 possible watermark waveforms and selecting the closest match). 

Several different types of predictive filters may be used to estimate the original, un- 
watermarked signal from the watermark signal. Consider a watermark application where a 
binary antipodal watermark signal W is scaled and then added to the host signal. In reading 
the watermark signal from a potentially distorted version of the watermarked signal, the 

20 objective is to reconstruct a binary antipodal signal representing the watermark signal from a 
potentially distorted version of the watermarked signal. An estimate of the original signal 
may be computed by filtering watermarked signal samples in a neighborhood around a 
sample of interest. One type of predictive filter is to compute an average of the pixels in the 
neighborhood (e.g., horizontally and vertically adjacent image samples, samples in a square 

25 neighborhood around the sample of interest, etc.). By subtracting a predicted sample value 
from the sample of interest, the demodulator produces a difference value with a polarity that 
estimates the polarity of the binary antipodal signal at that sample position. 

To generalize the watermark embedder example, one can view the modulation 
operations as functional blocks BO and Bl as shown in Fig. 11. The details of the 
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modulation functions within the functional blocks can vary from one implementation to 
another. 

Fig. 12, for example, shows the modulation function for the example shown in Fig. 9, 
rewritten so as to separate the spreading function from the modulation functions. Fig. 13 
5 shows the modulation functions within each of the functional blocks BO and Bl . By 

observation, one can see that the two modulation functions affect mutually exclusive sets of 
samples in the host signal. 

In view of the properties of the multi-carrier embedder illustrated in Figs. 9, 12 and 
13, it may be implemented in a watermark embedder as follows. The message to be 
10 embedded has two components f0,m and fl,n. Each of these components is a binary 
sequence. The same or different messages may be encoded in these sequences. The 
embedder employs a spreading function, in particular an Exclusive OR function, that spreads 
the message symbols in these binary sequences over a pseudo random carrier sequence. 
The resulting binary sequences include a set of chips that are mapped to locations 
15 within the host signal via a scatter pattern. In the case of an image watermark embedder 
employing the carrier functions depicted in Fig. 4, the locations in the host image are 2 by 2 
squares of image samples at a predetermined resolution. These locations are preferably 
spread throughout an image or portion of an image to be embedded. The watermark message 
may be repeated in several portions of the host signal, e.g., chunks or blocks of host signal 
20 samples or transform coefficients. Each of these portions may employ the same or different 
scatter patterns. 

The embedder employs random carrier assignments and parity multiplier assignments 
for each 2 by 2 square. The carriers, parity multipliers and pseudo random sequence values 
are each in the form of a binary antipodal signal. As such, they may be combined into a 
25 single table that maps input message sample values to a corresponding watermark sample 
values in a watermark signal. 

A watermark decoder compatible with this encoding scheme may be implemented as 
illustrated in Fig. 10 and described above. Because the message has at least two carriers, 
there is flexibility in the manner in which the decoder can exploit this attribute to read a 
30 message. If one message is modulated at a low frequency carrier signal (e.g., a DC carrier) 
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and another message is modulated at one or more higher frequency carriers, the decoder can 
selectively read messages at each of the different carrier frequencies. In some applications, 
the message at the low frequency carrier may survive better. In these applications, the 
decoder can perform a low pass read operation to extract the message conveyed via the low 
5 frequency carrier. In particular, the decoder filters and downsamples the watermarked signal 
to a low resolution that still encompasses the low frequency carrier. It then demodulates the 
watermark message with the low frequency carrier signal. Alternatively, the low frequency 
carrier may be affected more by the unmarked image, so only the high frequency carriers 
need to be read. 

10 To demodulate messages conveyed at each of the carriers, the decoder may employ 

correlation with the corresponding carrier signals. One form of correlation is to perform 
matched filtering with a reference signal representing the carrier signal, e.g., calculate a dot 
product of the carrier with the watermark signal. 

Concluding Remarks 

1 5 Having described and illustrated the principles of the technology with reference to 

specific implementations, it will be recognized that the technology can be implemented in 
many other, different, forms. To provide a comprehensive disclosure without unduly 
lengthening the specification, applicants incorporate by reference the patents and patent 
applications referenced above. 

20 The methods, processes, and systems described above may be implemented in hardware, 

software or a combination of hardware and software. For example, the watermark encoding 
processes may be incorporated into a watermark encoding system implemented in a programmable 
computer or a special purpose digital circuit. Similarly, watermark decoding, including watermark 
detecting and reading a watermark message, may be implemented in software, firmware, hardware, or 

25 combinations of software, firmware and hardware. The methods and processes described above may 
be implemented in programs executed from a system's memory (a computer readable medium, such 
as an electronic, optical or magnetic storage device). Additionally, content encoded with watermarks 
may be distributed on packaged media, such as optical disks, flash memory cards, magnetic storage 
devices, or distributed in an electronic file format. The watermarks may be decoded from content 

30 stored on machine readable media, including electronic, optical, and magnetic storage media. Image 
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watermarks may be embedded in images placed on physical objects, such as paper, ID cards, 
packaging, etc. and may be extracted from electronic images captured of these objects. Audio 
watermarks may be encoded in a manner that survives digital to analog and analog to digital 
conversion. 

5 The particular combinations of elements and features in the above-detailed 

embodiments are exemplary only; the interchanging and substitution of these teachings with 
other teachings in this and the incorporated-by-reference patents/applications are also 
contemplated. 



